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ABSTRACT 

Decision  Support  Systems  have  been  evolving  for  over  a  decade  to  assist  an 
individual  or  organization  in  the  decision  making  process.  DSS  are  often  difficult  to 
justify  or  evaluate  because  many  of  the  benefits  that  they  will  provide  are  intangible 
benefits  and  are  difficult  to  measure.  Over  the  past  decade,  there  have  been  several 
methods  published  that  can  assist  in  evaluation  or  justification. 

In  this  thesis,  I  propose  a  framework  designed  to  assist  the  user  in  selecting 
the  best  available  method  for  evaluation  or  justification  for  his/her  organization. 
The  framework  considers: 

•  The  goals  and  objectives  of  the  DSS  and  the  organization 

•  The  tangibility  of  the  goals  and  benefits 

•  The  need  or  requirement  to  quantify  goals  and  benefits 

DSS  are  continually  evolving  and  are  gaining  importance  in  the  information 
systems  area.  Evaluation  and  justification  will  become  increasingly  more  impor- 
tant. The  framework  will  provide  a  basis  for  selecting  the  most  appropriate  method 
available  for  an  organization. 
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I.  INTRODUCTION 

A  successful  DSS  can  provide  an  organization  the  needed  advantage  when 
trying  to  compete  in  an  environment  that  is  financially  burdened.  The  problem 
with  DSS  is:  "How  does  an  organization  justify  spending  dollars  on  a  DSS?" 

To  justify  a  DSS,  several  methods  have  been  presented  throughout  literature 
over  the  past  decade.  I  have  designed  a  framework  that  will  assist  the  user  in 
deciding  what  method  is  most  appropriate  for  the  user's  organization. 

This  framework  is  designed  to  take  the  user  through  a  process  of  three  steps 
or  questions  that  pertain  to  the  organization's  objectives  and  the  purpose  of  the 
proposed  DSS.  By  following  the  framework,  an  organization  can  select  the  most 
appropriate  method  available  for  not  only  justifying  a  DSS,  but  also  for  evaluat- 
ing a  current  DSS.  This  framework  is  flexible  and  can  expand  to  incorporate  new 
justification  or  evaluation  methods  if  and  when  they  become  available. 

A.     WHAT  IS  A  DECISION  SUPPORT  SYSTEM 

Decision  Support  Systems  (DSS)  are  defined  as:  "A  class  of  information  sys- 
tems that  draw  on  transaction  processing  systems  and  interacts  with  other  parts  of 
the  overall  information  system  to  support  the  decision  making  activities  of  managers 
and  other  knowledge  workers  in  organizations."  [Sprague  and  Carlson,  1982]  Other 
definitions  differ  between  what  an  actual  DSS  is  and  what  the  composition  of  a  DSS 
is.  Some  definitions  are  very  restrictive  to  the  criteria  that  involve  composition  of  a 
DSS,  while  others  are  of  a  broader  scope. 

Restrictive:  "DSS  are  an  interactive  computer-based  system  that  help  decision 
makers  utilize  data  and  models  to  solve  unstructured  problems."  [Sprague  and 
Carlson,  1982] 
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Broad:   "DSS  are  any  systems  that  make  some  contribution  to  decision  mak- 
ing."  [Sprague  and  Carlson,  1982] 


Both  of  these  types  of  definitions  have  drawbacks.  The  restrictive  definition 
greatly  limits  the  DSS  category,  whereas  the  broad  definition  includes  any  avail- 
able information  system.  A  viable  DSS  actually  falls  somewhere  between  these  two 
definitions. 

Decision  Support  Systems  (DSS)  have  been  evolving  for  the  past  decade.  The 
purpose  of  DSS  is  to  provide  a  decision  maker  with  the  resources  needed  for  assis- 
tance in  the  decision- making  process.  The  information  obtained  from  the  DSS  lends 
credibility  to  the  final  decision,  but  does  not  become  the  final  decision.  The  final 
decision  involves  the  DSS  data,  as  well  as  other  factors,  and  is  arrived  at  by  the 
decision  maker.  One  strength  of  a  DSS  is  that  it  can  focus  on  semi-structured  and 
unstructured  problems  that  are  not  normally  addressed  by  traditional  information 
systems. 

These  points  are  supported  by  Hogue  and  Watson  [1983],  as  listed  in  their 
criteria  for  a  DSS  as: 

•  Supports  but  does  not  replace  decision  making. 

•  Directed  toward  unstructured  and  or  semi-structured  decision 
making  tasks. 

•  Data  and  models  organized  around  the  decision(s). 

•  Easy  to  use  software  interface. 

•  Interactive  processing. 


TABLE  1.1:  DECISION  TABLE 

TYPE  OF  DECISION  PERCENT  OF  RESPONDENTS  WHOSE 

PRIMARY  USE  OF  THE  DSS  IS  FOR 
THIS  TYPE  OF  DECISION 

General  Long  Range  Planning  38% 

Strategic  Assessment  23% 

Product  Strategy  8% 

Negotiations  of  budgets  8% 

Reporting  and  Analysis  8% 

Operational  Planning  and  Control  6% 

Acquisition  Strategy  2% 

Capital  Investment  Strategy  2% 

Financial  Strategy  2% 

General  Budgeting  2% 

Cash  Flow  Management  2% 


•  DSS  use  and  control  is  determined  by  the  user. 

•  Flexible  and  adaptable  to  changes  in  the  environment  and  the  decision  maker's 
style. 

•  Quick  ad  hoc  DSS  building  capabilities. 

B.  DSS  DECISIONS 

A  DSS  can  assist  the  decision  maker  with  many  types  of  decisions  depending 
on  the  emphasis  of  the  specific  DSS.  In  a  survey,  [Meador,  et  al.,  1984],  a  DSS  was 
primarily  used  to  support  the  following  types  of  decisions  (See  Table  1.1): 

C.  DSS  BENEFITS 

Numerous  studies  addressed  the  benefits  that  an  effective  DSS  could  provide. 
[Hogue,  et  al.,  1985;  Keen,  1981;  Meador,  et  al.,  1984;  Money,  et  al.,  1988].  In  one 


survey  response,  eight  top  level  executives  (with  little  or  no  DSS  experience)  listed 
benefits  that  they  perceived  to  be  the  most  useful:  [Alavi,  1982] 

DSS  BENEFITS  IN  DESCENDING  ORDER  OF  USEFULNESS 

•  Provide  information  processing  and  retrieval  capabilities 

•  Evaluate  alternatives 

•  Assist  in  identifying  problems 

•  Assist  in  interpreting  information 

•  Provide  real  time  analysis  of  current  problems  and  opportunities 

•  Suggest  decision  alternatives 

•  Provide  ability  to  ask  "what  if"  questions 

•  Manage  executive  time  by  scheduling  daily  activities 

•  Increase  decision  confidence 

This  list  does  not  include  all  of  the  benefits  that  can  be  derived  from  an  effective 
DSS.  However,  it  is  important  to  understand  what  information  top  level  management 
would  like  to  see  when  justifying  a  DSS. 

D.     MOTIVATING  FACTORS 

What  motivates  organizations  to  procure  a  DSS  to  assist  in  the  decision  making 
process?  Benefits  are  listed  in  many  articles  that  advocate  DSS,  but  is  this  enough  to 
justify  the  investment?  [Hogue  and  Watson,  1983]  surveyed  18  major  corporations 
and  listed  the  motivating  factors  which  led  to  the  development  of  DSS. 


•  Accurate  Information  was  cited  by  12  corporations. 

•  Organizational  Champion  was  cited  by  8  corporations. 

•  New  Information  was  cited  by  6  corporations. 

•  Managerial  Mandate  was  cited  by  4  corporations. 

•  Timely  Information  was  cited  by  3  corporations. 

•  Cost  Reduction  was  cited  by  1  corporation. 

While  costs  and  tangible  benefits  are  crucial  to  an  organization,  these  findings 
on  motivation  show  how  important  intangible  benefits  have  become  in  the  DSS 
arena. 

E.  WHY  DO  ORGANIZATIONS  NEED  DSS? 

DSS  can  provide  numerous  benefits  ranging  from  increased  decision  alterna- 
tives to  improved  communication.  DSS  that  provide  increased  decision  alternatives 
could  save  (or  earn)  thousands  or  even  millions  of  dollars  simply  by  providing  the 
decision  maker  with  possible  alternatives  which  may  have  been  overlooked.  This  is 
just  one  example  of  a  potential  benefit.  DSS  can  also  provide  sensitivity  analysis 
functions  to  assist  a  decision  maker  in  choosing  between  alternatives.  The  list  of 
benefits  is  long  and  the  value  of  a  successful  DSS  could  be  priceless. 

F.  WHY  DO  ORGANIZATIONS  NEED  TO  EVALUATE  THEIR  DSS? 

While  several  DSS  have  become  successful,  others  have  been  failures.  DSS 
are  only  as  good  as  their  input.  While  it  is  easy  to  pick  up  a  magazine  and  read 
about  a  successful  DSS,  it  is  not  as  easy  to  find  articles  on  DSS  that  were  failures. 
Some  organizations  invest  in  DSS  which  ultimately  fail.  Thus,  there  must  be  some 
method  to  evaluate  the  success  of  an  organization's  DSS. 


Evaluation  and  feedback  are  very  important  to  any  operation.  Before  an  or- 
ganization implements  a  DSS,  an  evaluation  process  should  be  in  place.  This  can 
help  determine  if  the  outcome  is  worth  the  investment.  Evaluations  should  be  done 
on  a  continuing  basis  and  then  the  combinations  of  the  evaluations  could  be  used 
to  help  justify  upgrade  a  DSS  when  the  time  comes  to  do  so. 

G.     SCENARIO 

The  Commanding  Officer  for  the  US  Army  Recruiting  Command  knows 
that  a  decision  support  system  can  help  decision  making  tasks  such  as: 

•  From  where  are  the  majority  of  recruits  for  the  army  coming? 

•  What  can  be  done  to  increase  recruiting  in  each  geographic  region? 

•  What  effect  will  increased  advertising  have  in  certain  regions? 

•  How  much  should  be  spent  on  advertising? 

The  Commander  would  like  to  procure  a  DSS  but  must  justify  the  expenditure 
to  his  supervisors.  Goals  for  the  DSS  are  to  increase  recruitment  and  to  increase 
the  overall  quality  of  the  individuals  that  the  U.S.  Army  recruits. 

Analysis  to  determine  proper  method  of  justification  for  this  DSS  will  be  pre- 
sented in  Chapter  IV. 

H.     SUMMARY 

Many  articles  have  been  written  about  the  methods  used  to  justify  or  evaluate 
decision  support  systems. 

Hogue  and  Watson's  survey  [1983]  of  18  corporations  also  lists  the  methods 
used  by  the  participants  to  justify  building  the  DSS  (See  Table  1.2): 

These  and  other  justification  methods  will  be  examined  in  Chapter  II. 


TABLE  1.2:  JUSTIFICATION  OF  DSS 

Justification  Method  Percent  of  Corporations 
Intuition  44% 

Value  Analysis  33% 

Cost  and  Hard  Benefits  17% 

Cost  and  Intuitive  Benefits  6% 

A  notation  in  the  Hogue  and  Watson  survey  states,  "Ninety- Four  percent  of 
the  corporations  surveyed  made  no  formal  attempt  to  measure  the  financial  impact 
of  the  DSS  after  it  became  operational." 

The  purpose  of  this  thesis  is  to  survey  the  existing  methods  for  DSS  justifi- 
cation and  evaluation  and  to  present  a  framework  to  assist  the  decision  maker  to 
decide  the  most  appropriate  method  available  to  achieve  the  corporate  needs. 


II.  JUSTIFICATION 

A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  provide  information  on  justification  methods 
that  are  available.  Justification  allows  for  the  prediction  of  future  requirements, 
benefits  and  costs  to  evaluate  the  procurement  of  a  DSS.  Several  methods  of  justifi- 
cation of  DSS  have  been  published  in  the  literature.  The  discussion  of  each  method 
is  not  intended  to  be  a  rewrite  of  the  original  author's  work.  Instead,  its  purpose 
is  to  give  a  user  some  background  information  on  each  method  and  to  provide  the 
user  with  the  original  author's  work  for  further  elaboration. 

B.  JUSTIFICATION  METHODS 

The  following  methods  are  described  in  the  DSS  literature: 

•  Excess  Tangible  Costs 

•  Value  Analysis 

•  Benefit  Profile  Chart 

•  Incremental  Analysis 

•  Aggregate  Benefit  Value 

•  Profit  Index 

•  Savings/Investment  Ratio 

•  Intuition 

•  Cost  Benefit  Ratio 


1.      Excess  Tangible  Cost 

The  Excess  Tangible  Costs  method  [Litecky,  1981]  is  used  to  quantify 
intangible  benefits.  The  technique  itselfis  simple,  but  there  are  assumptions  which 
should  be  met  before  using  it: 

1.  Tangible  costs  and  benefits  are  relatively  easy  to  estimate. 

2.  Intangible  benefits  are  most  difficult  to  estimate. 

3.  Tangible  costs  are  ordinarily  much  greater  than  tangible  benefits. 

4.  Intangible  costs  are  insignificant. 

The  steps  involved  in  this  method  are: 

•  Step  1.  Summarize  all  of  the  tangible  costs  (i.e.,  program  development,  in- 
vestment in  equipment,  etc.). 

•  Step  2.  Analyze  the  sum  of  all  tangible  benefits  (i.e.;  cost  displacement  bene- 
fits, decreased  operating  costs,  etc.). 

•  Step  3.  Subtract  the  tangible  benefits  from  the  tangible  costs. 

This,  according  to  the  assumptions,  will  result  in  "Excess  Tangible  Costs". 
The  Excess  Tangible  Costs  is  an  important  figure  to  management  because  it 
is  the  amount  which  must  be  overcome  by  the  intangible  benefits. 

•  Step  4-  List  the  intangible  benefits. 

•  Step  5.  Derive  a  discount  factor  for  each  intangible  benefit.  The  discount 
factor  should  reflect  the  probability  that  the  particular  benefit  will  result  in 
overcoming  one  tangible  dollar. 


•  Step  6.  Multiply  the  discount  factor  (probability)  by  the  Estimated  Monetary 
Benefit  (EMB).  This  EMB  is  the  amount  the  benefit  would  achieve  if  there 
was  100%  probability  of  achieving  success. 

The  EMB  and  the  discount  factors  are  derived  via  a  feasibility  study. 
Example 


Tangible  Costs  ($25,000) 

-Tangible  Benefits  5,000 


Excess  Tangible  Costs  ($20,000) 

Intangible  Benefits 

Benefit  #1(10,000)  x  Discount  Factor(  .3)  $3,000 

Benefit  #2(10,000)  x  DF  (.5)  $5,000 

Benefit  #3(20,000)  x  DF  (.4)  $8,000 

Benefit  #4(25,000)  x  DF  (.3)  $7,500 

Total  Intangible  Benefits  $23,500 

NET $3,500 

Therefore,  a  combination  of  tangible  and  intangible  benefits  will  offset  the  tangible 
costs  by  a  net  amount  of  $3,500. 

The  term  Excess  Tangible  Costs  relates  to  the  dollar  amount  that  must 
be  overcome  by  the  intangible  benefits. 
2.     Value  Analysis 

The  Value  Analysis  method  was  first  introduced  by  Keen  [1981]  and  was 
divided  into  eleven  steps  by  Smith  [1983].  The  eleven  steps  are: 
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•  Step  1.  Define  the  benefits  to  be  obtained  if  a  prototype  system  is  developed. 

•  Step  2.  Determine  the  amount  users  are  willing  to  pay  to  obtain  the  benefits. 

•  Step  3.  Determine  whether  a  prototype  can  be  implemented  within  the  "cost 
threshold"  established  by  the  user. 

•  Step  4-  Design  the  prototype  and  measure  its  use  and  cost. 

•  Step  5.  Review  and  extend  benefits  list  if  necessary. 

•  Step  6.  Define  computer  hardware  and  software  which  would  permit  a  more 
complete  system  to  be  evolved  from  the  prototype. 

•  Step  7.  Determine  cost  of  expanding  the  system. 

•  Step  8.    Ask  users  to  determine  level  of  benefits  which  must  be  obtained  to 
justify  investing  in  an  expanded  system. 

•  Step  9.     Assuming  users  provide  a  feasible  new  "cost  threshold'1,  design  a 
second  version  of  the  system. 

•  Step  10.   After  the  second  version  is  implemented,  measure  its  use  and  costs 
and  determine  new  cost  threshold  for  a  possible  third  stage  of  development. 

•  Step  11.    Continue  steps  5-10  until  users  and  systems  designers  are  satisfied 
that  the  best  solution  has  been  achieved  under  existing  constraints. 

This  method  allows  the  users  to  determine  what  benefits  are  important 
and  place  a  monetary  figure  to  each  benefit.  The  monetary  values  are  based  on 
what  the  user  be  willing  to  pay  to  obtain  a  specific  benefit?  This  approach  uses 
the  "quick  hit"  approach  when  developing  a  DSS  and  utilizes  prototypes  to  get  a 
working  model  in  the  users'  hands  as  quickly  as  possible  and  at  a  low  initial  cost. 

11 


3.     Incremental  Analysis 

Incremental  Analysis  [Smith,  1983]  methodology  forecasts  the  probable 
need  of  a  DSS  or  Decision  Tool,  and  then  determines  the  various  ways  to  accomplish 
these  items  without  building  or  buying  the  DSS.  "The  first  step  in  using  incremental 
analysis  is  to  predict  future  workloads  and  to  assign  probabilities  to  the  forecasted 
workload.  Systems  personnel  then  determine  alternative  methods  for  accomplishing 
the  workload  and  assign  costs  to  each  alternative."   [Smith,  1983] 

Example 

A  marketing  firm  believes  there  is  a  70%  chance  that  they  will  need  a  DSS 
in  the  next  two  years  to  assist  in  determining  a  marketing  strategy  for  geographic 
regions  and  advertising  dollars  needed  to  promote  the  strategy. 

The  alternative  is  that  there  is  a  30%  chance  that  a  DSS  will  not  be 
needed. 

Problem:  Determine  the  alternative  ways  to  accomplish  the  same  goal  and  deter- 
mine costs  for  each. 

1.  Hire  additional  personnel  to  determine  marketing  strategy. 

2.  Subcontract  to  an  outside  bureau. 


DSS  Needed      DSS  Not  Needed     Total  Expected 
(J) (J5) Cost 

Hire  Marketing     $150,000/year  0  $105,000 

Personnel 

Using  Outside      $100,000/year  0  $  70,000 

Consultants 
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From  this  analysis,  the  least  costly  alternative  is  to  hire  an  outside  con- 
sultant at  $70,000  per  year.  If  an  appropriate  DSS  be  acquired  for  less  than  $70,000 
per  year,  the  firm  should  explore  either  buying  or  building  a  DSS. 
4.      Profitability  Index 

The  Profitability  Index  method  is  simply  an  accounting  exercise  used  to 
determine  a  project's  worth.  The  profitability  index  is  a  ratio  that  accounts  for  cash 
inflows  into  a  project  and  the  amount  of  investment  required.  [Garrison  1988]  It 
takes  into  account  only  the  tangible  costs  and  benefits  that  occur  with  a  project.  In 
most  cases,  DSS  provide  intangible  benefits.  This  method  would  be  inefficient  if  an 
organization  is  interested  in  any  intangible  benefits. 

The  formula  for  profitablity  index  is: 

Present  Value  of  Cash  Inflows 
Investment  Required 

The  higher  the  profitability  index,  the  more  desirable  the  project.    The 
profitablity  index  it  allows  for  more  than  one  project  to  be  evaluated  against  other 
projects  that  may  or  may  not  provide  the  same  service. 
Project  A 


Hw      i  o  1        $250,000  Est.  cash  inflow 

ire  Marketing  Personnel        tiso,ooo  invctmait 

Required 

Profitability  Index  =  1.67 
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Project  B 


$150,000  Est.  cash  inflows 


Purchase  of  a  Marketing  DSS  s70,ooo  investment 

Required 


Profitability  Index  =  2.14 


Using  this  simple  example  shows  how  Project  B  has  the  higher  Profitabil- 
ity Index,  thus,  it  should  be  selected  over  Project  A. 
5.      Benefit  Cost  Ratio 

The  Benefit  Cost  Ratio  [Naval  Data  Automation  Command  (NDAC), 
1980]  can  be  used  when  costs  and  benefits  are  unequal.  To  determine  the  economic 
desirability  of  an  investment,  the  benefits  are  divided  by  the  costs,  providing  a  single 
figure  for  comparison.  The  value  represents  the  amount  of  benefits  obtained  per  unit 
of  cost. 

npn   Quantifiable  Output  Measure 

—  Uniform  Annual  Cost 

•  (UAC)  Uniform  Annual  Cost  =  Pre^^ue 

•  (QOM)  Quantifiable  Output  Measures  =  Tangible  Benefit 

[Economic  Analysis  Procedures  for  ADP,  NDAC,  1980] 

The  terms  b^  and  b\f  are  factors  taken  from  Appendix  B.  They  are  determined  by 

the  project  life. 

Example 

Investment  decisions  occur  randomly  over  the  course  of  the  year.  The 
present  manual  system  for  evaluating  an  investment  decision  takes  approximately  3 
days  (24  man-hours).    An  Investment  DSS  can  analyze  investment  decisions  more 
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rapidly,  allowing  a  manager  to  complete  the  process  in  one  day  (8  man-hours). 
Over  the  year  many  investment  decisions  are  overlooked,  due  to  the  inability  of  the 
investment  department  to  keep  pace. 

QOM  manual  =  121  Investment  Analysis  Opportunities  per  year 

QOM  DSS  =  365  Opportunities  per  year 

Cost  Category  Manual  DSS 

One-time  (year  one)  0  $50,000  (DSS) 

Recurring  (year  2-9)     $50,000  (salary)     $50,000  (salary) 


TT4^  ,       50,000  6.042 -.954        254,400 

UAC  manual  =       ,—— ±- -  =     -  '         =  $50,000 

6.042  -  .954  5.088 


50, 000(.954)  + 50, 000(6.042 -.954)       302, 100 
UAC  DSS  = 6.042  -  .954 =  ^88"  =  $5M75 


BCR  manual  =  ff^f     121      =  .00242 
UAC  50,000 


The  proposed  DSS  has  a  higher  BCR  than  the  current  manual  system. 
Therefore,  it  is  the  more  cost  effective  alternative. 
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6.     Aggregate  Benefit  Value 

The  Aggregate  Benefit  Value  (ABV)  method  [NDAC,  1980]  is  one  of  many 
weighted  methods  that  are  available.  Weighted  methods  may  be  more  applicable  in 
the  DSS  arena  due  to  the  fact  that  many  of  the  benefits  are  intangible.  ABV  works 
similarly  to  the  Cost  Benefit  Ratio,  method  discussed  above. 

The  example  for  the  ABV  method  uses  the  same  UAC  figures  derived 

from  the  previous  example,  with  the  major  difference  that  no  QOM  can  be  derived. 
Example: 

The  vice-president  wants  you  to  evaluate  a  DSS  and  compare  it  to  your 

present  manual  system.     His  major  concerns  are  the  need  for  a  faster  response 

time  to  unexpected  investment  opportunities  and  the  ability  to  answer  "what  if" 

scenarios.    Other,  less  important  factors  include  better  use  of  resources  and  more 

effective  teamwork.  (See  Tables  2.1  and  2.2) 

Using  this  method,  the  DSS  again  appears  more  efficient  because  it  has  a  higher 
BCR. 

*  The  factor  weights  and  the  rankings  are  determined  prior  to  the  evaluation.  The 
factor  weights  are  determined  by  the  user  depending  on  the  value  the  user  has  placed 
on  each  decision  factor.  The  rankings  are  then  determined  by  system  personnel, 
normally  on  a  scale  of  1  to  10.  (Faster  response  time  is  three  times  higher  for  the 
DSS  as  compare  to  the  manual  method,  thus  the  rankings  of  9  for  the  DSS  and  only 
3  for  the  manual  method.) 

The  Aggregate  Benefit  Ratio  allows  a  user  to  factor  in  the  intangible 
benefits,  although  they  are  difficult  to  quantify.  However,  caution  must  be  exercised 
to  prevent  the  introduction  of  personal  opinions  and  biases.  There  always  exists  the 
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TABLE  2.1:  BENEFIT  RANKING  DSS 


Decision  Factor 
Faster  Response  Time 
Ability  for  "What  If"  scenarios 
Better  Use  of  Resources 
Teamwork 
TOTAL 

UAC  (59,375  as  determined  in  CBR  example) 
BCR 


*Factor  Weight     *Ranking     Product 


27 


3 

7 

21 

2 

8 

14 

2 

6 

12 

74 

5.9375 

12.46 

TABLE  2.2:  BENEFIT  RANKING  MANUAL 


Decision  Factor 
Faster  Response  Time 
Ability  for  "What  If"  scenarios 
Better  Use  of  Resources 
Teamwork 
TOTAL 

UAC  (50,000  as  determined  in  CBR  example) 
BCR 


'Factor  Weight     ^Ranking     Product 


3 

5 

15 

2 

5 

10 

2 

5 

10 

44 

5.0000 

8.8 
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possibility  of  adjusting  factor  weight  or  rankings  as  needed.  Adjustments  will  bring 
about  a  different  conclusion. 

7.      Savings/Investment  Ratio  -  (SIR) 

A  Savings/Investment  Ratio  (SIR)  [NDAC,  1980]  is  the  relationship  be- 
tween future  cost  savings  and  the  investment  necessary  to  obtain  those  savings.  A 
SIR  of  1  indicates  that  the  present  value  of  the  savings  is  equal  to  the  present  value 
of  the  investment.  For  an  investment  to  be  economically  sound,  the  SIR  value  must 
be  greater  than  one.  SIR  is  a  characteristic  of  cost  only  and  does  not  consider 
benefits  (except  cost  savings). 

SIR  is  used  when  evaluating  an  alternative  against  the  present  situation. 

Example: 

Presently  you  are  the  head  of  an  investment  decision  department  that 
consists  of  four  employees  and  four  personal  computers.  The  department  is  respon- 
sible for  analyzing  investment  opportunities  for  a  local  bank.  The  four  employees  are 
paid  $40,000  each  per  year  as  a  salary.  The  bank  has  the  opportunity  to  purchase 
an  Investment  DSS  for  $50,000.  The  Investment  DSS  would  decrease  the  workload 
of  your  department  by  25%,  allowing  you  to  reassign  one  of  your  four  employees  to 
a  different  department  and  saving  $40,000  per  year  in  salaries  that  you  would  have 
had  to  pay  to  hire  another  investment  analyst.  The  DSS  has  a  two  year  service 
guarantee  but  possibly  could  be  obsolete  after  2  years. 

..„      PVS       $40,000,,  $72,840 

*sm  m  TV,  =  mM(Lm)  =  $50^00  =  L4568 

Since  the  SIR  is  greater  than  1.0,  the  investment  is  economically  sound. 
The  present  value  of  the  DSS  is  greater  than  the  present  value  of  the  cost. 


18 


PVS  is  the  present  value  of  the  savings.  (All  future  savings  must  be 
discounted  by  using  factors  from  Appendix  B  to  bring  the  value  to  a  present  day 
figure.) 

PVI  is  the  present  value  of  the  investment.  (If  the  investment  is  a  one  time 
investment  that  is  paid  in  full  there  is  no  need  to  discount  the  amount.  However,  if 
the  investment  occurs  over  a  specified  period  of  time  the  the  factors  from  Appendix 
B  should  be  applied.) 
*  Project  Year  Discount  Factors  such  as  1.821  can  be  found  in  Appendix  B. 

8.  Benefit  Profile  Chart 

The  Benefit  Profile  Chart  method  [Smith,  1983]  uses  a  comprehensive 
checklist  of  benefits  (See  Appendix  A)  used  by  system  professionals  to  help  in 
preparing  justification  documents.  The  checklist  can  be  adapted  to  any  organi- 
zation's needs.  By  graphically  displaying  the  benefit  charts,  a  system  professional 
can  better  justify  a  DSS  package  to  top  executives.  An  example  of  a  Benefit  Profile 
Chart  can  be  found  in  Appendix  A. 

9.  Intuition 

One  of  the  most-used  strategies  is  simply  the  use  of  intuition.  [Hogue 
and  Watson,  1983]  Watson  found  through  an  empirical  study  that  44%  of  the  re- 
spondents to  his  questionnaire  used  intuition  over  any  other  method  when  justifying 
procurement  of  a  DSS. 

Also  along  these  lines  is  the  motivation  to  procure  a  DSS  because  the 
competition  has  employed  one.  Many  organizations  are  motivated  by  successful 
decision  support  systems  that  have  been  employed  by  like  organizations.  The  greater 
degree  of  success  from  an  organization's  competitors,  the  larger  the  motivation  factor 
for  procuring  a  DSS.  In  Hogue  and  Watson's  survey  [1983],  organizational  champion 
was  cited  by  44%  of  the  corporations  as  a  motivating  factor  for  procuring  a  DSS. 
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III.  EVALUATION 

A.  INTRODUCTION 

A  current  operating  DSS  should  be  evaluated  with  respect  to  its  effectiveness, 
costs,  and  benefits.  This  chapter  provides  information  on  the  evaluation  methods 
that  are  available.  The  discussion  of  each  method  is  not  intended  to  be  a  rewrite  of 
the  original  author's  work.  Its  purpose  is  to  give  a  user  some  background  information 
on  the  method  and  reference  to  the  original  author's  work  for  further  investigation. 

B.  EVALUATION  METHODS 

The  following  methods  are  described  in  the  DSS  Literature: 

•  Return  on  Investment  (ROI) 

•  Expected  Value 

•  Value  Analysis 

•  ROI/w  Value  Analysis  or  User  Satisfaction 

•  Profit 

•  User  Satisfaction 

1.     Return  on  Investment 

The  Savings/ Investment  Ratio  [Garrison,  1988]  can  be  used  to  calculate 
the  actual  return  on  investment.  The  savings/ investment  ratio  is  the  relationship 
between  cost  savings  and  the  initial  investment  with  any  operating/maintenance 
costs,  to  obtain  those  savings.    A  SIR  of  1  indicates  that  the  present  value  of  the 
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savings  is  equal  to  the  present  value  of  the  investment.  For  an  investment  to  be 
economically  sound,  the  SIR  must  be  greater  than  1.  The  SIR  ratio  is  a  characteristic 
of  cost  only  and  does  not  consider  benefits  (except  cost  savings). 

SIR  is  used  when  evaluating  a  new  system  against  the  current  one. 

Example: 

The  investment  decision  department  invested  in  a  DSS  three  years  ago. 

The  chief  of  the  department  would  like  to  determine  if  the  investment  was  profitable. 

The  DSS  the  department  utilizes  had  allowed  them  to  eliminate  one  of  the  paid 

analyst  postions.   The  cost  savings  from  saved  salaries  is  $40,000  for  the  first  year 

and  42,000  and  43,000  in  the  second  and  third  years.   In  the  fourth  year  the  DSS 

became  obsolete  because  of  several  changes  in  the  surrounding  environment.    The 

department  was  then  forced  to  hire  another  analyst.  The  DSS  had  provided  several 

intangible  benefits,  including  increased  alternative  choices  and  faster  response  time. 

The  initial  investment  was  $50,000  with  maintenance  and  upgrade  costs  of  $15,000 

over  the  four  years. 

Savings  $125,000 

Dirt  =  — =    -,  =  i.y^ 

Investment        3>65,000 

Since  the  SIR  is  greater  than  1.0,  the  DSS  was  a  sound  investment. 

This  DSS  returned  nearly  twice  the  investment.  This  is  using  only  tan- 
gible costs.  The  fact  that  there  were  also  associated  intangible  benefits  makes  it 
even  more  successful.  The  fact  that  the  DSS  was  a  sound  investment  although 
it  eventually  became  obsolete,  could  be  sufficient  justification  for  investigating  the 
procurement  of  a  new  DSS. 

Return  on  Investment  could  also  be  calculated  using  the  formula:  [Polokoff, 
1990] 

Benefits  —  Depreciation 


ROI  = 


Costs 
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The  DSS  in  the  above  example  had  a  4-year  expected  life  (for  deprecia- 
tion purposes),  but  in  the  fourth  year,  the  department  hired  another  analyst,  thus 
providing  no  cost  savings  for  the  fourth  year.  The  the  figures  would  show: 

Benefits  =  S2jb2ffi  =  $31  250 

Depreciation  =  ^00  =  $12,500 

Costs  =  $65,000  (50,000  initial  +  15,000  maintenance) 

ROI  per  year  =  S3h2^~^'500  =  .2885X100%  =  28.85% 

ROI  =  28.85% 


The  organization's  standards  would  dictate  if  the  ROI  was  an  acceptable 


amount. 


Other  accounting  methods  are  also  available  for  use  in  determining  prof- 
itability of  a  project.  The  major  point  to  keep  in  mind  with  these  methods  is  that 
intangible  benefits  are  overlooked. 
2.     Profit 

Profit  [Garrison,  1988]  is  an  area  that  can  be  used  to  evaluate  the  success 
of  a  DSS,  but  only  after  an  extended  period  of  time.   The  DSS  could  operate  for 
several  years  before  anyone  would  be  able  to  attribute  the  change  in  profit  level  to 
the  success  of  the  DSS. 
Example  1: 

A  successful  organization  (without  the  aid  of  a  DSS)  enjoys  a  profit  of 
one  million  dollars  per  year  over  three  years.  A  DSS  is  then  built  to  help  them  pick 
better  investment  opportunities.  The  first  year  profits  increase  to  1.2  million.  The 
second  and  third  year  profits  increase  to  1.3  and  1.4  million,  respectively. 

It  could  be  assumed  that  the  organization  appears  to  be  turning  a  bigger 
profit  due  to  the  use  of  the  DSS.  However,  a  part  of  the  increase  could  be  due  to 
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many  other  factors  which  would  have  brought  about  the  increase  without  the  DSS's 
involvement.  The  true  test  then  would  be  to  operate  without  the  use  of  the  DSS 
and  to  compare  profit  levels.  Most  organizations  would  not  attempt  this  strategy. 
The  best  way  to  evaluate  a  DSS  using  this  method  would  entail  a  thorough  analysis 
of  all  the  environmental  factors  that  could  possibly  lead  to  increasing  or  decreasing 
profits. 
Example  2: 

A  successful  organization  (without  the  aid  of  a  DSS)  enjoys  a  profit  of 
one  million  dollars  per  year  over  three  years. 

They  then  build  a  DSS  to  help  them  pick  better  investment  opportunities. 
The  first  year  their  profits  decrease  to  .9  million.  The  second  and  third  year  profits 
decrease  to  .8  and  .7  million,  respectively. 

The  organization  now  appears  to  be  losing  profits  due  to  the  DSS. 

This  situation  is  easier  to  remedy.  If  the  DSS  was  scrapped  or  rebuilt, 
several  more  years  would  need  to  pass  to  check  the  profit  levels  to  see  if  an  increase 
results. 

However,  environmental  factors  could  be  causing  the  decreased  profit.  In 
this  example,  it  is  easier  to  doubt  the  effectiveness  of  the  DSS.  Whether  or  not  the 
profits  increase  or  decrease,  it  would  take  a  long  time  to  determine  whether  the  DSS 
has  been  successful  using  this  method. 
3.     Expected  Value 

The  accurate  use  of  this  method  [Smith,  1983]  requires  an  experienced 
and  knowledgeable  person  in  the  field  of  statistics,  economics,  systems  analysis  and 
DSS  to  follow  these  steps: 
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•  Step  1:    List  the  DSS  benefits  and  assign  them  to  a  group  of  possible  cost 
reduction  benefits  (Group  A,  8,  C.  .  .column  1). 

•  Step  2:   Assign  a  possible  cost  reduction  range  (column  2)  and  a  probability 
of  occurrence  (column  3)  to  each  group. 

•  Step  3:  Multiplying  the  midpoint  of  the  assigned  range  (column  4)  times  the 
total  cost  of  the  project  (assign  the  product  to  column  5). 

•  Step  4'  Then  multiply  the  probability  of  occurrence  times  (column  5)  to  arrive 
at  the  probable  savings  that  is  listed  in  (column  6). 

Example 

Group  Benefit 


Group  A     Increased  number  of  alternatives 
New  insight  and  learning 
Improved  communications 
Control 

Group  B     Make  better  use  of  data 

Better  understanding  of  business 

Group  C     Ability  to  carry  out  ad-hoc  analysis 

Fast  response  to  unexpected  situations 
Better  decisions 


The  actual  cost  measured  in  dollars  from  this  example  was  $1,000,000. 
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1 

2 

3 

4 

5 

6 

Grp 

Poss.  Cost 

Probability 

Midpoint 

Cost 

Probable 

Reduction 

of 

Reduction 

Savings 

Occurrence 

C4  x  Total  Cost 

C3xC5 

A 

0-10% 

70% 

5.0% 

$50,000 

$35,000 

B 

10-15% 

50% 

12.5% 

$125,000 

$62,500 

C 

15-25% 

40% 

20.0% 

$200,000 

$80,000 

Total  Probable  Savings 

$177,500 

Using  this  method,  it  is  shown  that  of  the  $1,000,000  in  actual  costs, 
$177,500  is  the  probable  savings  from  the  intangible  benefits  alone. 
4.      Value  Analysis 

Value  Analysis  [Keen,  1981]  was  discussed  in  Chapter  II.  However,  value 
analysis  can  also  be  used  to  evaluate  a  DSS.  Once  a  DSS  has  been  up  and  running 
in  an  organization,  the  question  may  exist  whether  to  update  or  scrap  the  DSS. 
Value  Analysis  allows  the  evaluator  to  determine  what  the  user  is  willing  to  pay  in 
order  to  keep  his  present  DSS  and  to  determine  if  there  are  other  benefits  that  can 
be  added  to  the  DSS. 

Steps  8  through  11  allow  the  organization  to  effectively  evaluate  their 
DSS  by  going  back  to  the  users  and  discussing  expansion  or  scraping  the  present 
DSS. 

•  Step  8  Ask  the  users  to  determine  level  of  benefits  that  must  be  obtained  to 
justify  investing  in  an  expanded  system. 

•  Step  9  Assuming  users  provide  a  feasible  new  "cost  threshold",  design  a  second 
version  of  the  system. 
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•  Step  10  After  the  second  version  is  implemented,  measure  its  use  and  costs 
and  determine  new  cost  threshold  for  a  possible  third  stage  of  development. 

•  Step  11  Repeat  steps  5-10  until  users  and  systems  designers  are  satisfied  that 
the  best  solution  has  been  achieved  under  existing  constraints. 

The  evolving  DSS  that  Value  Analysis  provides  is  an  excellent  way  of 
continually  evaluating  an  organizations  DSS.  If  users  are  pleased  with  the  initial 
versions  of  the  DSS,  the  organization  can  continually  update  the  DSS  to  include 
more  functions. 

5.      User  Satisfaction 

Several  methods  are  available  to  measure  user  satisfaction.  [Baroudi  and 
Orlikowski,  1988]  However,  a  common  method  is  the  use  of  a  questionnaire- type 
survey.  Surveys  can  help  the  organization  determine  the  features  of  a  DSS  which 
are  the  most  beneficial,  used,  or  successful  and  it  can  also  help  determine  those 
features  that  are  worthless  or  dissatisfying.  There  are  cases  where  DSS  tools  and 
user  satisfaction  surveys  have  not  produced  consistent  results.  These  studies  showed 
that  user  satisfaction  surveys  can  sometimes  be  deceiving.  In  an  assessment  [Aldag 
and  Power,  1986]  of  computer-assisted  decision  analysis,  users  thought  favorab  of 
the  decision  tool,  however  using  the  decision  tool  did  not  improve  their  decisions.  In 
a  separate  study  [Gallupe  and  DeSanctis,  1988],  the  decision  tool  greatly  enhanced 
the  quality  of  the  decision,  however,  satisfaction  and  confidence  were  lower  than  the 
group  that  did  not  use  the  decision  tool.  Despite  these  findings,  user  satisfaction 
surveys  are  still  widely  used  and  accepted.  Most  surveys  use  a  Likert-type  scale, 
allowing  the  respondent  to  evaluate  on  a  scale  of  1  to  7,  the  features  of  the  DSS.  Ap- 
pendix C  shows  an  example  of  a  User  Satisfaction  Survey.  [Baroudi  and  Orlikowski, 
1988] 
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6.     ROI  w/Value  Analysis  or  User  Satisfaction 

This  method  combines  two  methods  [Baroudi  and  Orlikowski,  1988]  [Gar- 
rison, 1988]  and  attempts  to  balance  the  tangible  benefit  side  (ROI)  with  the  intan- 
gible benefits  (Value  Analysis  or  User  Satisfaction). 

The  methods  have  been  described  above.  An  organization  could  run 
both  methods  to  determine  the  bottom  line  return  on  their  investment  and  to  also 
give  consideration  to  intangibles  by  implementing  value  analysis  or  user  satisfaction 
techniques.  By  implementation  of  this  method,  the  organization  would  evaluate 
their  present  DSS  and  would  also  provide  data  for  future  justification  to  updated 
or  modify  the  current  DSS.  By  implementing  value  analysis  or  user  satisfaction 
techniques,  it  will  be  easier  to  overcome  a  lower  than  expected  ROI  (provided  that 
the  user  satisfaction  level  was  high  with  the  initial  DSS). 

This  method  help  to  satisfy  those  who  want  to  see  a  bottom  line  figure, 
while  at  the  same  time  taking  into  account  the  intangible  benefits  that  are  always 
present  in  the  DSS  arena. 
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IV.  A  FRAMEWORK  FOR  DSS 
EVALUATION  AND  JUSTIFICATION 

A.     INTRODUCTION 

There  are  several  ways  to  evaluate  or  justify  an  investment.  Procurement  of 
any  system,  whether  a  DSS  or  a  traditional  information  system,  should  be  thought 
of  as  an  investment.  This  chapter  presents  a  framework  that  will  assist  a  user  in 
selecting  an  appropriate  method,  or  set  of  methods,  to  evaluate  or  justify  a  DSS. 
The  framework  that  I  have  designed  is  a  multi-level  decision  tree.  (Figure  4.1) 

The  decision  tree's  purpose  is  to  guide  the  user  through  3  levels  of  questions 
to  arrive  at  a  methodology  which  will  assist  his/her  organization  when  evaluating 
or  justifying  a  DSS.  The  questions  within  the  decision  tree  are  designed  to  separate 
the  available  methods.  The  answers  to  the  questions  within  the  decision  tree  should 
be  based  on  the  user's  organizational  goals.  Upon  answering  the  questions,  the  user 
will  arrive  at  a  methodology  which  will  best  fit  organizational  needs. 
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Figure  4.1:  Framework  For  Evaluating  or  Justifying  DSS 
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B.  LEVEL  1 

The  first  level  of  the  framework  addresses  the  purpose  of  the  DSS  assessment. 
Is  the  user  trying  to  justify  future  procurement  (buy  or  develop)  a  DSS,  or  is  he 
looking  to  evaluate  the  current  DSS  in  his  organization?  Several  methods  are  avail- 
able to  justify  and  to  evaluate  a  DSS,  however,  most  methods  apply  only  to  either 
justification  or  evaluation. 

C.  LEVEL  2 

The  second  level  in  this  framework  deals  with  the  objectives  to  be  accomplished 
within  the  DSS.  Are  the  goals  mostly  tangible  goals  (e.g.,  time  savings,  cost  savings, 
profit)  or  are  they  mostly  intangibles  (e.g.,  better  decisions,  more  alternatives,  user 
satisfaction,  improved  service  for  the  customer)? 

The  key  to  answering  these  questions  is  in  knowing  the  organization's  mission 
or  goals.  If  an  organization's  goal  is  strictly  increased  profit,  this  goal  is  relatively 
easy  to  quantify.  However,  while  increased  profit  is  the  primary  goal,  many  or- 
ganizations have  secondary  goals.  If  the  organization  wants  increased  profits  with 
increased  organization  morale,  the  goals  are  not  as  easily  quantifiable. 

When  answering  the  Level  2  question,  the  adjective  "mostly"  should  be  con- 
sidered. 

If  the  organization's  primary  goals  are  mostly  tangible  with  some  intangible 
secondary  goals,  then  it  should  take  the  the  "tangible  route."  If  the  organization's 
primary  goals  are  mostly  intangible  with  tangible  secondary  goals,  then  the  "intan- 
gible route"  should  be  followed. 

In  the  case  where  the  organization's  goals  are  unknown  or  the  user  cannot 
decide  between  tangible  and  intangible,  the  route  label  "both  or  unknown"  should 
be  chosen.    However,  this  route  will  require  the  user  to  do  a  thorough  analysis  of 
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many  methods  to  justify  the  DSS  in  order  to  find  the  method  that  best  fits  the 
organization. 

Many  DSS  are  geared  toward  functional  areas  and  may  be  far  removed  from 
the  primary  goals  of  an  organization.  In  this  case,  the  functional  area  goals  need  to 
be  addressed  in  determining  the  most  appropriate  route  in  the  framework. 

D.      LEVEL  3 

The  third  level  of  the  proposed  framework  leads  the  user  to  a  method  or  a 
set  of  available  methods  considered  to  be  the  most  appropriate  to  the  user's  given 
situation. 

The  following  routes  are  available  for  DSS  Justification. 

1.     Justifying/Tangibles 

If  a  user  is  trying  to  justify  purchasing  or  developing  a  DSS  and  the 
benefits  are  mostly  tangible,  the  third  question  is  "Do  intangible  benefits  have  any 
impact  on  goals?"  If  the  intangible  benefits  do  have  some  impact,  then  the  "yes" 
route  will  lead  to  the  following  methods  of  justification: 

•  Excess  Tangible  Cost  [Litecky,  1981] 

•  Value  Analysis  [Keen,  1981] 

If  the  intangible  benefits  have  little  or  no  impact  on  goals,  the  "no"  route 
will  lead  to  the  following  methods: 

•  Profitability  Index  [Garrison,  1988] 

•  Cost  Benefit  Ratio  [NDAC,  1980] 

•  Savings  Investment  Ratio  [NDAC,  1980] 

•  Incremental  Analysis  [Smith,  1983] 
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2.  Justifying/Intangibles 

If  the  user  is  trying  to  justify  a  DSS  and  the  goals  are  mostly 
intangible,  the  question  to  be  addressed  will  be:  "Is  there  a  need  to 
quantify  intangible  benefits?"  An  organization  may  desire  a  DSS  that  provides  in- 
formation for  better  decisions  (intangible),  however,  it  wants  to  show  a  "bottom  line 
figure"  (tangible).  The  following  methods  are  available  to  help  give  the  organization 
that  "bottom  line  figure": 

•  Excess  Tangible  Costs  [Litecky,  1981] 

•  Aggregate  Benefit  Value  (weighted  methods)  [NDAC,  1980] 

If  the  organization  does  not  require  a  bottom  line  figure,  the  following 
methods  are  more  appropriate: 

•  Value  Analysis  [Keen,  1981] 

•  Benefit  Profile  Chart  [Smith,  1983] 

•  Intuition  [Hogue  and  Watson,  1983] 

•  Success  Stories 

These  methods  are  discussed  in  Chapter  II. 

3.  Evaluate/Tangible 

When  evaluating  a  DSS  designed  for  mostly  tangible  benefits  with  some 
intangible  impact  on  goals,  Return  on  Investment  in  conjunction  with  User  Satis- 
faction or  Value  Analysis  would  be  appropriate. 

If  the  intangible  benefits  do  not  affect  goals,  simply  Return  on  Investment 
(Actual)  or  Profit  would  be  better  measures. 
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4.     Evaluating/Intangibles 

If  an  organization  is  evaluating  a  DSS  designed  for  intangible  benefits  and 
there  is  still  a  need  to  quantify  the  answer,  then  the  Expected  Value  Method  would 
be  appropriate. 

If  there  is  not  a  need  to  quantify  an  evaluation,  the  following  methods 
would  be  more  appropriate: 

•  User  Satisfaction  [Baroudi  and  Orlikowski,  1988] 

•  Value  Analysis  [Keen,  1981] 

E.     SUMMARY 

Organizations  may  have  any  number  of  goals.  An  organization  may  have  a 
goal  that  is  tangible  in  nature  but  the  user  desires  a  DSS  that  provides  intangible 
benefits  (e.g.,  more  alternatives  to  help  produce  better  decisions).  The  objective  of 
the  DSS  now  falls  into  the  realm  of  intangibles  even  though  the  organization's  goals 
are  tangible.  To  successfully  use  the  decision  tree,  both  the  goals  of  the  organization 
and  the  proposed  purpose  of  the  DSS  must  be  taken  into  account  and  the  appropriate 
measures  applied  as  listed  above. 

What  follows  are  three  scenarios  which  demonstrate  the  usefulness  of  the  pro- 
posed framework. 
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F.     SCENARIO  1 

The  Commanding  Officer  for  the  U.S.  Army  Recruiting  Command  knows  that 
a  decision  support  system  can  help  him  with  several  decision  making  tasks  such  as: 

•  From  where  are  the  majority  of  recruits  for  the  U.S.  Army 
coming  from? 

•  What  can  be  done  to  increase  recruitment  in  each  geographic 
area? 

•  What  effect  will  increased  advertising  have  in  certain 
regions? 

•  How  much  should  be  spent  on  advertising? 

The  commander  would  like  to  procure  a  DSS,  but  must  justify  the 

expenditure  to  his  superiors.  Goals  for  the  DSS  are  to  increase  recruitment 
and  to  increase  the  overall  quality  of  individuals  that  the  U.S.  Army  recruits. 

In  this  scenario  the  best  response  would  be: 
Level  1.  Justification 

The  organization  would  like  to  procure  a  DSS.  The  DSS  will  require  justifica- 
tion. 
Level  2.  Tangible 

The  scenarios  goals  are  for  increased  recruitment  and  increased  quality  of 
recruits.  These  goals  are  tangible  and  can  be  measured.  The  recruiting  command 
administers  a  aptitude  test  to  judge  overall  quality  of  the  recruit.  These  goals  can 
be  measured  using  traditional  Cost  Benefit  Analysis  techniques. 
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Level  3. 

The  answer  to  the  level  three  question  is  "No",  intangibles  benefits  do  not 
impact  on  goals. 

The  scenario  does  not  state  any  intangible  benefits  as  either  primary  or  sec- 
ondary goals. 

This  information  leads  the  analysis  to  the  following  applicable  alternative  jus- 
tification methods: 

•  Incremental  Analysis  [Smith,  1983] 

•  Profitability  Index  [Garrison,  1988] 

•  Cost  Benefit  Ratio  [NDAC,  1980] 

•  Savings/Investment  Ratio  [NDAC,  1980] 
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G.     SCENARIO  2 

The  human  resource  manager  controls  pay  and  suggests  appropriate  increases 
for  each  position  within  the  organization.  He  would  like  a  DSS  to  assist  in  the 
decision-making  process.  The  DSS  would  be  a  great  timesaver,  but  would  primarily 
be  used  to  satisfy  the  organization's  members  and  help  keep  morale  at  a  high  level. 

The  DSS  would  have  to  be  able  to  assist  in  dividing  up  the  budget  dollar  for 
annual  pay  raises  on  a  semi-annual  and  annual  basis.  Keeping  employees  happy  is 
an  important  goal  since  to  alienate  them  would  contribute  to  the  loss  of  outstanding 
employees. 

The  DSS  should  be  able  to  capture  market  information  on  wnat  other  organi- 
zations are  paying  (this  can  be  accomplished  through  a  financial  service). 

The  DSS  should  be  able  to  incorporate  the  consumer  price  index.  It  should  be 
able  to  show  where  the  organization  stands  against  the  market  on  current  salaries. 

It  should  be  able  to  answer  questions  such  as  "What  if  I  want  to  increase  all 
programmers'  salaries  in  my  organization  to  account  for  the  difference  in  their  pay 
and  the  market  leader?  How  much  would  this  increase  costs  to  the  organization?" 

Top  management  would  like  the  tangible  benefits  quantified  and  a  list  of  all  of 
the  intangible  benefits  that  the  DSS  will  provide,  but  there  is  not  a  need  to  attempt 
to  quantify  improved  morale  and  other  intangible  benefits. 

In  this  scenario,  the  best  response  would  be: 
Level  1.  Justification 

The  organization  would  like  to  procure  a  DSS  which  requires  justification. 
Level  2.  Intangible 

The  DSS  would  be  used  primarily  to  assist  in  the  decision  making  process,  to 
save  valued  time  and  to  keep  organization's  members  motivated.  The  scenario  lists 
several  intangible  goals  but  makes  no  mention  of  intangible  goals. 
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Level  3. 

The  answer  to  the  level  three  question  is  "No"  there  is  not  a  need  to  quantify 
the  intangibles  benefits). 

Top  management  would  like  the  tangible  benefits  quantified,  but  the  intangible 
benefits  should  be  listed.  There  is  no  need  to  place  a  specific  dollar  value  to  the 
intangible  benefits. 

This  information  leads  the  human  resource  manager  to  these  applicable  alter- 
native justification  methods: 

•  Value  Analysis  [Keen,  1981] 

•  Benefit  Profile  Chart  [Smith,  1983] 

•  Intuition  [Hogue  and  Watson,  1983] 
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H.     SCENARIO  3 

A  major  airline  is  currently  running  a  decision  support  system  for  assisting 
allocation  of  planes  to  different  flights  and  scheduling  passengers  for  each  flight. 
The  DSS  assists  the  decision-maker  in  insuring  that  planes  are  available  for  every 
flight.  If  a  plane  breaks  down,  the  DSS  can  find  another  available  plane  to  meet  the 
grounded  plane's  commitments. 

The  primary  goal  of  the  DSS  is  to  increase  the  airline's  profits  through  in- 
creased customer  satisfaction.  The  better  the  organization's  ability  to  meet  all  of 
its  commitments,  the  more  satisfied  the  customers  should  be.  The  assumption  is 
made  that  when  an  airline  has  to  cancel  flights  because  of  breakdowns,  customers 
will  begin  to  switch  to  a  more  reliable  carrier. 

Management  wants  to  know  if  an  upgrade  of  the  present  system  is  needed. 
While  the  DSS  in  place  is  doing  a  satisfactory  job,  there  is  a  question  as  to  whether 
a  newer  system  would  be  better. 

In  this  scenario,  the  best  response  would  be: 
Level  1.  Evaluation 

In  the  scenario,  the  airline  presently  has  a  DSS  and  would  like  to  evaluate  the 
DSS  to  assist  the  airline  in  making  decision  on  whether  to  invest  more  money  on 
upgrading  their  system. 
Level  2.  Tangible 

The  goals  in  this  scenario  are  centered  around  increasing  the  airlines  profits. 
Level  3. 

Yes  (Intangibles  Benefits  have  an  impact  on  goals) 

The  primary  goal  is  to  increase  profits.  However,  customer  satisfaction  does 
have  an  impact  on  the  goals  of  increased  profits. 
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This  information  leads  the  organization  to  the  following  analysis: 

•  ROI  w/Value  Analysis 

•  ROI  w/User  Satisfaction 
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V.  CONCLUSION 

A.  THERE  IS  A  NEED  FOR  DSS 

Decision  Support  Systems  can  play  a  vital  link  in  the  area  of  decision  making. 
DSS  are  not  intended  to  make  final  decisions,  rather  to  assist  the  decision  maker  in 
the  decision  making  process.  Expanding  a  decision  makers  views  and  alternatives, 
combined  with  the  ability  to  run  sensitivity  analysis  on  each  alternative,  can  greatly 
enhance  the  quality  of  a  decision.  However,  not  every  DSS  is  successful.  There  must 
be  some  means  set  in  place  to  justify  and  evaluate  a  DSS.  Even  if  the  evaluation 
is  nothing  more  than  a  user  satisfaction  survey,  this  is  still  a  method  which  will 
provide  feedback  to  top-level  management.  Without  some  sort  of  feedback  method, 
the  organization  will  simply  run  blind. 

In  Chapter  III,  several  methods  of  justification  and  evaluation 
are  presented.  Each  method  has  advantages  and  disadvantages. 

B.  COST  BENEFIT  METHODS 

"Cost/benefit  evaluations  will  probably  provide  the  most  meaningfu'  ^sults  to 
management."  [Sprague  and  Carlson,  1982]  However,  CBA  runs  into  trouble  when 
trying  to  evaluate  or  justify  intangible  benefits. 

C.  VALUE  ANALYSIS 

This  method  is  simple  to  implement,  using  prototyping  which  can  help  limit 
the  initial  investment  while  testing  the  water  to  see  if  the  system  will  be  a  success. 
However,  it  is  "less  rigorous  than  cost  benefit  techniques  and  may  not  include  all 
the  measures  that  are  relevant."  [Sprague  and  Carlson,  1982] 
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D.  USER  ATTITUDE/SATISFACTION 

Using  surveys  to  measure  user  satisfaction  can  also  run  into  problem  areas. 
[Sprague  and  Carlson,  1982]  lists  the  following  problem  areas  when  using  question- 
naires: 

1.  In  developing  a  questionnaire  to  measure  attitudes,  one  may  be  trying  to 
quantify  that  which  is  not  quantifiable  and  the  subsequent  analysis  may  be 
misleading. 

2.  Questionnaire  respondents  may  interpret  questions  differently  than  intended, 
and  answers  to  some  questions  may  influence  others. 

3.  Administering  questionnaires  may  be  an  inconvenience  to  individuals  and  ex- 
pensive (in  terms  of  hours  lost  by  the  respondents  and  hours  spent  by  the 
interviewers). 

4.  The  method  does  not  identify  the  causes  of  any  measured  changes. 

E.  OTHER  METHODS 

•  Expected  Value 

•  Incremental  Analysis 

•  Excess  Tangible  Costs 

•  Aggregate  Benefit  Value 

Depending  on  the  organization  and  its  goals,  these  methods  can  be  very  ben- 
eficial and  simple  to  use.  However,  they  tend  to  rely  on  subjective  judgement  when 
establishing  weighting  or  probability  of  occurrence  figures.  In  some  cases,  the  slight- 
est change  in  weights  or  probabilities  can  drastically  change  the  outcome  of  the 
evaluation. 
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F.     USE  OF  THE  DECISION  TREE 

The  main  purpose  of  this  thesis  was  to  provide  a  framework  that  can  be  used  to 
assist  the  decision  maker  in  determining  the  most  appropriate  method  for  evaluating 
or  justifying  a  DSS.  The  decision  tree  that  is  provided  in  Chapter  IV  (along  with  a 
description  of  the  methods)  allows  a  decision  maker  to  quickly  assess  the  methods 
that  best  suits  his  personal  or  organizational  needs.  The  methods  for  evaluating 
and  justifying  a  DSS  will  continually  change.  The  decision  tree  can  be  incorporated 
to  handle  any  changes.  The  basic  premise  for  the  decision  tree  is  based  on  three 
questions  that  the  organization  must  answer: 

1.  Is  the  decision  maker  attempting  to  justify  future  procurement  of  a  DSS  or  is 
the  decision  maker  attempting  to  evaluate  the  current  DSS? 

2.  Are  the  goals  of  the  organization  mostly  tangible  or  intangible?  If  the  DSS  is  a 
personal  DSS  for  an  individual,  then  the  individual's  goals  should  be  analyzed. 
Hopefully,  the  individual  decision  makers  goals  complement  the  organizational 
goals. 

3.  If  the  goals  are  mostly  tangible,  do  the  intangible  benefits  have  any  impact  on 
obtaining  the  organizational  goals? 

4.  If  the  goals  are  mostly  intangible,  is  there  a  need  to  quantify  intangible  bene- 
fits? 

By  successfully  answering  these  questions,  the  decision  tree  should  lead  the 
decision  maker  to  a  method(s)  that  will  provide  the  best  available  results  given  the 
organization's  views  and  goals  and  prove  acceptable  to  top-level  management. 
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G.     SIX  POINTS 

While  attempting  to  justify  or  evaluate  a  DSS,  the  following  points  should  be 
kept  in  mind: 

1.  DSS  can  provide  both  tangible  and  intangible  benefits.  All  appropriate  bene- 
fits should  be  considered  when  the  justification  process  begins. 

2.  Benefits  should  be  viewed  in  light  of  the  organizational  objectives. 

3.  The  use  of  a  personal  DSS  should  be  viewed  in  light  of  the  user's  objectives, 
and  should  coincide  with  that  of  the  organization. 

4.  Evaluation  of  a  DSS  is  an  important  feedback  mechanism  that  helps  to  provide 
information  to  top-level  management. 

5.  Evaluations  throughout  the  lifecycle  of  the  DSS  are  necessary  to  minimize  the 
risks  of  continually  pouring  money  into  an  unsuccessful  system. 

6.  Doing  one  or  two  things  well  is  far  better  than  doing  several  things  that  are 
mediocre.  Do  not  try  to  provide  a  DSS  that  delivers  everything  because  it  can 
truly  cost  everything. 
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APPENDIX  A 

Cost  Benefit  Profile  Chart  [20] 

Intangible  benefits  of  MIS/DSS  leading  to  improved  business  performance. 

•  Category  A  =  Some  improvement  in  existing  system. 

•  Category  B  =  Significant  improvement  in  existing  system. 

•  Category  C  =  Significant  new  benefit. 
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Business  Function/Activity 

Degree  of  Performance  Improvement 

Engineering/Research 

A 

B 

C 

Interactive  Problem  Solving  within 
Company 

Interactive  Problem  Solving  with 
Customer 

Stimulation  of  new  ideas 
(i.e.,  graphics) 

Faster  Design  (i.e.,  computer-aided 
design) 

Control  of  Specifications/drawing 

Access  to  technical  information 

Processing  of  engineering  change 
orders 

Manpower/Project  Management 

Management  of  Professional's  time 
e.g.,  reduced  clerical  workload 
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Business  Function/Activity 

Degree  of  Performance  Improvement 

Finance/ Accounting 

A 

B 

C 

Budget  Preparation 

Use  of  operating/leverage 

Privacy  of  data  information 

Security  of  data  information 

Integrity  (accuracy  of  data) 

Planning/Control  of  liquid  assets 

Capital  Budgeting 

Auditing  and  Internal  Control 

Simplified  Reporting 

Timely  Reports 
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Business  Function/Activity 

Degree  of  Performance  Improvement 

Employee  Relations/Human  Resources 

A 

B 

C 

Identification  of  best  performers 
(Individual  and  Groups) 

Strategic  Manpower  Planning 

Places  and  methods  for  recruiting 

Improved  government  reporting 

Monitoring  of  EEO/ERISA/OSHA 
Standards 

Higher  motivation  for  workforce 

—  Career  planning  capabilities 

—  Turnover/absenteeism 

—  Fringe  benefits  planning/control 

—  Job  satisfaction 

Employee  Training 

e.g.,  computer  assisted  instructions 
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Business  Function/Activity 

Degree  of  Performance  Improvement 

Employee  Relations/Human  Resources 

A 

B 

C 

Understanding  of  how  HRM  functions 

Wage  and  Salary  Planning  and  Control 
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Business  Function/Activity 

Degree  of  Performance  Improvement 

General  Management 

A 

B 

C 

Increased  Communications  among 
departments 

Planning  data  more  quickly/easily 
assessible 

Ability  to  provide  "what  if"  reports 

Faster  development  of  new  systems 

Easier  to  use  system 

Increased  secretarial  efficiency, 
effectiveness 

Better  Meetings 

Provides  better  reliability 

Improved  Accuracy/Conciseness 
Timeliness  relevance  of  all 
information 

Cost  Avoidance 

—  Precludes  need  to  hire  new  people 
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Business  Function/Activity 

Degree  of  Performance  Improvement 

General  Management 

A 

B 

C 

—  Need  fewer  computer  programs 

—  Need  less  program  maintenance 

—  Reduced  travel  costs 

—  Better  use  of  programmers  time 

Business  Function/Activity 

Degree  of  Performance  Improvement 

Decision  Support  System 

A 

B 

C 

—  Goal  seeking 

—  What  if  stimulation 

—  Graphics/modeling 

The  Cost  Benefit  profile  is  not  meant  to  fit  every  aspect  of  a  particular  organization.  It 
must  be  tailored  to  a  specific  organication.  This  is  strictly  a  guideline  of  a  Cost  Benefit 
Profile. 
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APPENDIX  B 

Project  Year  Discount  Factors 


TABLE  A  TABLE  B 

Present  Value  of  $1  (Single  Present  Value  of  $1  (Cumulative 

Amount:  Used  when  cash  flows  Uniform  Series:  To  be  used 

accrue  in  varying  amounts  when  cash  flows  accrue  in  the 

each  year)  same  amount  each  year) 


Project  10%  10% 


1  0.954  0.954 

2  0.867  1.821 

3  0.788  2.609 

4  0.717  3.326 

5  0.652  3.977 

6  0.592  4.570 

7  0.538  5.108 

8  0.489  5.597 

9  0.445  6.042 

10  0.405  6.447 

11  0.368  6.815 

12  0.334  7.149 

13  0.304  7.453 

14  0.276  7.729 

15  0.251  7.980 


Note:  Table  B  factors  represent  the  cumulative  sum  of  Table  A  factors  through  any 
given  project  year. 

Reference:  [NDAC,  1980] 
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APPENDIX  C 

User  Satisfaction  Survey 

This  appendix  provides  a  basic  user  satisfaction  survey  [Baroudi  and  Or- 
likowski,  1988].  The  survey  is  not  intended  to  meet  every  organizations  needs. 
However,  it  does  provide  the  basis  for  a  user  satisfaction  survey,  which  should  be 
tailored  to  an  organization's  needs  and  goals. 

This  survey  should  be  administered  to  the  users  of  the  DSS  after  weights  have 
been  placed  on  each  of  the  possible  responses.    Likert  scale  normal  runs  from  1 
to  7  points.    It  is  up  to  the  organization  to  establish  an  acceptable  score  prior  to 
administering  the  survey. 
Please  follow  these  instructions: 

1.  Check  each  scale  in  the  position  that  describes  your  evaluation  of  the  factor 
being  described. 

2.  Check  each  scale,  do  not  omit  any. 

3.  Check  only  one  position  for  each  scale. 

4.  Check  in  space,  not  between  spaces. 

ANSWER  BASED  ON  YOUR  OWN  FEELINGS 
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1.  Relationship  with  the  Management  Information  Department  (MID)  staff 


dissonant 


hannonius 


bad 


good 


2.  Processing  of  requests  for  changes  to  existing  DSS 


fast 


slow 


untimely      : : : : : : : :      timely 


3.  Degree  of  training  provided  to  users 


complete 


incomplete 


low 


:  high 


4.  User  understanding  of  the  DSS 


insufficient 


insufficient 


complete 


incomplete 
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5.  User's  feeling  of  participation 

positive  : : : : : : : :      negative 

insufficient         : : : :_   _:_   _:_   _:_   _:      sufficient 


6.  Attitude  of  the  Management  Information  Department  staff 

cooperative        : : : : : : : :      belligerent 

negative  : : : : : : : :      positive 


7.  Reliability  of  output  information 

high  : : : : : : : :      low 

superior        : : : : : : : :      inferior 


8.  Relevancy  of  output  information  (Does  the  DSS  do  intended  function) 


useful  :_  _:_   _:      _:_   _:_  _:_   _:_   _:      useless 


relevant  :_  _:_   _:_  _:_   _:_  _:_   _: :      irrelevant 
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9.  Accuracy  of  output  information 


inaccurate 


accurate 


low 


high 


10.  Precision  of  output  information 


low 


:      high 


definite 


uncertain 


11.  Communication  with  the  Management  Information  Department  staff 


dissonant 


harmonious 


destructive      : : : : : : : :      productive 


12.  Time  required  for  the  new  systems  development 


unreasonable 


reasonable 


acceptable 


unacceptable 
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13.  Completeness  of  the  output  information 


sufficient 


insufficient 


adequate 


inadequate 


14.  DSS  assisted  in  decision  making  process 


definite 


uncertain 


superior 


inferior 


15.  Is  DSS  easy  to  learn  and  use 


easy 


difficult 


fast 


slow 


16.  List  the  functions  that  are  the  most  beneficial: 


17.  List  the  functions  that  are  most  needed: 
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